# Product Tests

## Running
```bash
# List available suites and environments
./bin/ptl suite list
./bin/ptl env list

# Run a product test suite
./bin/ptl suite run --suite <suite-name>

# Run a product test group
./bin/ptl test run --environment <env-name> -- -g <group-name1>,<group-name2>

# Run a specific test
./bin/ptl test run --environment <env-name> -- -t <fully-qualified-method-name1>,<fully-qualified-method-name2>

Building

After changing product test code, rebuild both modules before running:

mvnd install -pl :trino-product-tests,:trino-product-tests-launcher -DskipTests -Dair.check.skip-all=true

The launcher copies the test jar and server-main into Docker containers, so they must be rebuilt too if required.

Infrastructure

• Tests run inside a tests Docker container on the ptl-network
• Trino coordinator container: presto-master (port 8080)
• Container names defined in EnvironmentContainers.java
• Environment providers live in trino-product-tests-launcher under env/environment/
• Suite definitions live in trino-product-tests-launcher under suite/suites/
• Test groups defined in trino-product-tests-groups/.../TestGroups.java
• Tempto configuration (JDBC connections, users) in trino-product-tests-launcher/src/main/resources/docker/trino-product-tests/

Framework

• Tests use Tempto (TestNG-based, NOT JUnit). See also Trino product tests README
• Tests extend io.trino.tempto.ProductTest and use @Test(groups = {...}) for grouping
• Queries run via JDBC through QueryExecutor (e.g. connectToTrino(), onPostgres())
• Dependency injection via Guice (@Inject, @Named("databases.trino.server_address"))
• Multiple user configs (alice, bob, etc.) defined per environment in tempto-configuration.yaml

Test setup: @BeforeMethodWithContext

• DO NOT use @BeforeClass or @BeforeMethod - Tempto Guice injection hasn't happened yet at that point
• Use @BeforeMethodWithContext (from io.trino.tempto) - runs after injection, before each test method
• For one-time setup, guard with private volatile boolean initialized (standard Trino pattern):

  private volatile boolean initialized;
  
  @BeforeMethodWithContext
  public void setUp()
  {
      if (initialized) {
          return;
      }
      // ... one-time setup
      initialized = true;
  }

Test groups and PROFILE_SPECIFIC_TESTS

• PROFILE_SPECIFIC_TESTS marks tests that require a specific environment/profile to run
• The default test runner excludes profile_specific_tests - tests in this group are skipped unless explicitly included
• Custom suites that provide the right environment should NOT use PROFILE_SPECIFIC_TESTS - the suite already guarantees the correct environment via its EnvironmentProvider
• Use PROFILE_SPECIFIC_TESTS only when the test also belongs to a broader group that runs across multiple environments (e.g. INSIGHTS tests that run in both generic and specific profiles)

Environment providers

• Live in trino-product-tests-launcher under env/environment/, annotated with @TestsEnvironment
• Extend EnvironmentProvider, compose base modules (e.g. StandardMultinode, Hadoop, Minio) via constructor injection
• Override extendEnvironment(Environment.Builder) to configure containers, copy files, set env vars
• Config files per environment: conf/environment/<env-name>/config-coordinator.properties etc., copied into containers via withCopyFileToContainer()

Suite definitions

• Live in trino-product-tests-launcher under suite/suites/, extend Suite
• Override getTestRuns(EnvironmentConfig) returning List<SuiteTestRun>
• Each SuiteTestRun pairs an environment with test groups: .withGroups(...), .withExcludedGroups(...), .withTests(...), .withExcludedTests(...)

Debugging

• View coordinator logs: docker logs presto-master
• View test container logs: docker logs tests
• Worker logs: docker logs presto-worker-1 (etc.)
• Filter tests with -t <fully.qualified.MethodName> or -g <group-name> to iterate quickly on failures

</details>
</summary>